In order to stay updated with fast-changing trends in technology, it may be helpful to keep a track of events or changes occurring with regard to the operation of systems implemented across various industries. For example, a system may relate to Internet of Things (IoT) applications and users may subscribe to obtain updates pertaining to any changes in such applications. From the perspective of service providers, they prefer to maintain their customer base by keeping them aware of their developments. Conventionally, there are monitoring applications available to interact with the systems and provide appropriate information to the users who have subscribed for the information. However, all events of a system may not be relevant to a user and, therefore, the user may not prefer receiving updates of all the events of the system.
Furthermore, a large amount of data traffic has to be handled to ensure that appropriate updates of the systems can be provided to the subscribers. Handling such large data traffic can lead to operational issues, as it may be difficult to manage the data traffic for such a large amount of information in order for the system to operate effectively. In a typical message pub-sub system, a publisher publishes messages to a topic and there may be one or more subscribers that are interested in such messages. A subscriber may register interest for all the messages or a subset of the messages specifying some filtering criteria. Such systems are typically implemented by routing all the messages of a topic to a process such as, for example, a broker process, and the broker process may evaluate subscription filters for each message and forward the matched messages to the corresponding subscriptions.
Such a system may work well with a low volume of messages and a low number of subscriptions. However, there may be scenarios where it may be beneficial to have thousands of subscriptions for a topic. For example, in case of a weather application where each weather application instance registers interest for a few places, if each application instance is modeled as a subscriber to a weather topic, then the broker process may match each incoming weather update against those registered weather subscriptions. However, as the weather application becomes more popular, the number of subscribers would go up and the computation required for filtering for each message may increase proportionately. Such computing limitations may also restrict a number of messages to be handled by a topic, even in the case of a lower number of subscriptions. In order to mitigate the computing limitation, the system may have multiple topics and limit each topic to a maximum number of subscriptions. However, even this feature may place additional computational burdens on both the publisher and subscriber application.
An event grid may be a backplane to provide events routing service that receive information about an event and publish the received information to the intended recipients. In an example, the event grid may manage routings of all the events coming from different sources and route them to different destinations. With the event grid, one can subscribe to an event occurring in a resource or event source and get notification for the same. Generally, then event grid may be organized in a cluster of event grid nodes. The events may be provided to the subscribers through event grid nodes which function independent of each other. An event grid node may be understood as a virtual machine where an event grid host runs. A publisher may send the events to any of the event grid nodes. Each node may receive published events, filter the received events to match the events to their intended recipients, and publish the matched events.
Furthermore, each region may have a set of clusters. A given region may be paired to another region that may be geographically near to the given region for availability, reliability, and load balancing at the region level. However, the publishers may send the events at variable rates that may lead to resource exhaustion in the event grid. Such publishers may impact the performance of the event grid for other publishers as well. In such situations, the nodes may be overloaded, under-loaded, or unavailable. Under-loading or over-loading of a node may indicate that the load is unbalanced among nodes and, therefore, hamper the performance of the event grid.
Additionally, owing to the large data traffic, there exists a possibility of breakdown of the monitoring application. Moreover, in case of such breakdown, the monitoring application may lose out on the data traffic during the duration of the breakdown and therefore, the users may not receive the updates. Also, when the subscriber is permanently unavailable, intermittently unavailable, intermittently failing, or permanently failing, the event cannot be delivered to the subscriber. There is therefore a need for technical solutions that scale an event notification system for large amounts of data while at the same time maintaining the reliability of the event notification system.